The present invention relates to apparatus and a method for enhancing the temporal processing of video signals which are received via a transmission channel and, in particular, to a system which imperceptibly encodes motion vectors in the transmitted signal which are then decoded at the receiver and used to enhance temporal processing of the received video signals.
Television receivers which use frame or field memories to improve the quality of images reproduced from a television signal are well known. One example of a system of this type is described in U.S. Pat. No. 4,665,437 entitled ADAPTIVE FIELD OR FRAME STORE PROCESSOR, which is hereby incorporated by reference for its teachings on video signal processing. The system described in this patent applies a frame comb filter or a line comb filter to separate the luminance and chrominance components of a received video signal. The line comb filter is used whenever time-base errors are detected in the received signal. Otherwise, the frame comb filter is used. This patent also describes a progressive scan system which switches between inter-field and intra-field systems responsive to time-base errors.
Another television system which uses temporal processing to enhance the quality of reproduced images is described in a paper by S. Naimpally et al. entitled "Integrated Digital IDTV Receiver With Features" IEEE Transactions on Consumer Electronics, Vol. 34, No. 3, August, 1988, pp 410-419, which is hereby incorporated by reference for its teachings on video signal processing.
The system described in the referenced paper employs a recursive temporal filter containing a frame delay element to reduce the level of noise in the reproduced image. An exemplary filter, of the type employed in the referenced paper, is shown in FIG. 1. In this Figure, the received luminance signal is applied to one input port of a subtracter 110, the other input port of which is coupled to receive a frame-delayed video signal from a frame delay element 122. The subtracter 110 subtracts the received signal from the delayed signal and applies the result to a scaling circuit 112. The scaling circuit multiplies the difference signal by a value K, which is changed in response to the magnitude of the difference between the previous frame and the current frame (i.e. motion in the underlying images).
The scaled difference signal is added, by a summing circuit 114, to the frame delayed signal provided by the delay element 122. The output signal of the summing circuit is the output signal of the temporal filtering system. This signal is also stored in the frame delay element 122 for use in processing video signals during the next successive frame interval.
The value of K is determined by applying the difference signal developed by the subtracter 110 to a low-pass filter 116 and then taking the absolute value of the low-pass filtered signal in a rectifying circuit 118. This filtered and rectified signal is applied as address values to a read-only memory (ROM) 120. The ROM 120 contains a look-up table (LUT) which specifies the value of K as a function of the rectified difference between the luminance signals in the current and previous frames. FIG. 2 is a graphical depiction of an exemplary function that may be stored in the ROM 120.
As shown in FIG. 2, K has a relatively small value (e.g. 1/8) when the difference between the current and previous frames is small (i.e. no motion) and a relatively large value (e.g. 1) when the difference between the current and previous frames is large.
The level of noise reduction in decibels (dB), NR, which may be achieved by the circuitry shown in FIG. 1 is given in the equation (1). EQU NR=10log.sub.10 ((2-K)/K) (1)
While the system described above is a motion adaptive noise reduction filter, other types of motion adaptive processors, for example, frame/field/line comb filters for separating the luminance and chrominance signal components, and intra-field/inter-field progressive scan systems, for reducing the visibility of image artifacts caused by the raster scan, may also be made motion-adaptive. In each of these systems, signals representing relatively still areas of the image are processed in longer time frames while signals representing moving areas of the image are processed in shorter time frames.
The problem with all simple motion adaptive processors of the type described above, is that there is a noticeable loss of picture quality in moving areas of the image while there is a significant improvement in still areas. In the case of noise reduction, if the K factor is not increased sufficiently for moving areas, the picture appears smeared in those areas and the edges of moving objects may appear blurred. If, however, the K factor is increased to be close to its maximum value, noise may appear in areas of moving detail, such as along the edges of moving objects. Motion artifacts of this type are also apparent in motion-adaptive interlace to progressive scan converters and motion-adaptive comb filters.
Significant improvement in performance can be obtained by using motion compensated processing instead of motion adaptive processing. Exemplary motion compensated systems are described in a paper by E. Dubois et al. entitled "Noise Reduction in Image Sequences Using Motion Compensated Temporal Filtering," IEEE Transactions on Communications, Vol. COM-32, No. 7, July 1984, pp 826-831 and in a section of a book by J. S. Lim entitled Two-Dimensional Signal and Image Processing Prentice-Hall Englewood Cliffs N.J., 1990, pp 497-498, 507-511, and 570-575. These references are hereby incorporated by reference for their teachings on motion-compensated video signal processing.
In a typical motion-compensated system, a received video image is divided into blocks of, for example, eight-by-eight pixels and each block is compared to the previous frame to find a similar set of pixels from that frame which most closely matches the block. A motion vector is associated with the block, indicating the displacement of the matching block from the previous frame.
During noise reduction processing, this matching block is provided by a frame memory in response to the block from the current image. With reference to FIG. 1, the motion block designated by the motion vector for the current block of the input signal is read from the frame memory 122 and applied to the subtracter 110 and adder 114 as the current block is processed. This operation results in temporal processing in the direction of motion. Images produced from these motion-compensated temporally processed signals exhibit greatly improved noise reduction even on moving pictures.
While it is technically possible to incorporate motion-compensated processing of this type in a consumer television receiver, at present, the cost associated with such a system would be very high. Accurate motion estimation requires a full search over a relatively large region of the previous frame. For real-time processing, a relatively large number of fast processors and a relatively large memory would be needed to calculate motion vectors of the type described in the above-referenced paper and book.